Setting the penalty for logistic and multinomial regression with glmnet #863

hfrick · 2023-01-25T21:51:45Z

closes #858

This PR moves where we set s, glmnet's argument for the penalty value, from inside the relevant multi_predict() method into the relevant predict_raw() method.

This means that now the default penalty value specified in the parsnip spec will be used when penalty = NULL, also for for type = "raw" (aka the original issue).

This also means that the call stack for multi_predict() for logistic and multinomial regression now follows that of linear regression and what happens in the code is what was laid out in the comments, e.g.,

parsnip/R/logistic_reg.R

Lines 209 to 234 in 2249cbb

    
           # ------------------------------------------------------------------------------ 
        
           # glmnet call stack for logistic regression using `predict` when object has 
        
           # classes "_lognet" and "model_fit" (for class predictions): 
        
           # 
        
           #  predict() 
        
           # 	predict._lognet(penalty = NULL)    <-- checks and sets penalty 
        
           #    predict.model_fit()               <-- checks for extra vars in ... 
        
           #     predict_class() 
        
           #      predict_class._lognet() 
        
           #       predict_class.model_fit() 
        
           #        predict.lognet() 
        
           # glmnet call stack for logistic regression using `multi_predict` when object has 
        
           # classes "_lognet" and "model_fit" (for class predictions): 
        
           # 
        
           # 	multi_predict() 
        
           #    multi_predict._lognet(penalty = NULL) 
        
           #      predict._lognet(multi = TRUE)           <-- checks and sets penalty 
        
           #       predict.model_fit()                    <-- checks for extra vars in ... 
        
           #        predict_raw() 
        
           #         predict_raw._lognet() 
        
           #          predict_raw.model_fit(opts = list(s = penalty)) 
        
           #           predict.lognet() 
        
           # ------------------------------------------------------------------------------

The tests are in extratests: tidymodels/extratests#72

library(parsnip)

data(lending_club, package = "modeldata")

lr_spec <- logistic_reg(penalty = 0.123) %>% set_engine("glmnet")
f_fit <- fit(lr_spec, Class ~ log(funded_amnt) + int_rate + term,
             data = lending_club)
predict(f_fit, lending_club[1:5, ], type = "raw")
#>         s1
#> 1 2.894019
#> 2 2.894019
#> 3 2.894019
#> 4 2.894019
#> 5 2.894019

data("penguins", package = "modeldata")
penguins <- tidyr::drop_na(penguins)

mr_spec <- multinom_reg(penalty = 0.123) %>% set_engine("glmnet")
f_fit <- fit(mr_spec, species ~ island + bill_length_mm + bill_depth_mm,
             data = penguins)

predict(f_fit, penguins[1:5,], type = "raw")
#> , , 1
#> 
#>      Adelie Chinstrap    Gentoo
#> 1 -5.131312 -7.544642 -7.542310
#> 2 -5.229012 -7.544642 -6.800678
#> 3 -5.424412 -7.544642 -7.142970
#> 4 -4.545112 -7.544642 -7.884602
#> 5 -5.180162 -7.544642 -8.626234

^{Created on 2023-01-25 with reprex v2.0.2}

so that it: - also gets applied in `predict(type = "raw")` - structure follows that of `linear_reg()`, which is also laid out in the comments

github-actions · 2023-02-21T01:04:57Z

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

hfrick added 2 commits January 25, 2023 21:26

setting penalty in predict_raw() method

1b33a6e

so that it: - also gets applied in `predict(type = "raw")` - structure follows that of `linear_reg()`, which is also laid out in the comments

sticking to the general pattern

b652e0d

hfrick mentioned this pull request Jan 25, 2023

glmnet engines for logistic_reg() and multinomial_reg() respect the penalty from the spec tidymodels/extratests#72

Merged

hfrick requested a review from topepo January 25, 2023 22:37

topepo approved these changes Feb 6, 2023

View reviewed changes

topepo merged commit a517c87 into main Feb 6, 2023

topepo deleted the glmnet-penalty-raw branch February 6, 2023 14:57

github-actions bot locked and limited conversation to collaborators Feb 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Setting the penalty for logistic and multinomial regression with glmnet #863

Setting the penalty for logistic and multinomial regression with glmnet #863

Uh oh!

hfrick commented Jan 25, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Feb 21, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	# ------------------------------------------------------------------------------
	# glmnet call stack for logistic regression using `predict` when object has
	# classes "_lognet" and "model_fit" (for class predictions):
	#
	# predict()
	# predict._lognet(penalty = NULL) <-- checks and sets penalty
	# predict.model_fit() <-- checks for extra vars in ...
	# predict_class()
	# predict_class._lognet()
	# predict_class.model_fit()
	# predict.lognet()


	# glmnet call stack for logistic regression using `multi_predict` when object has
	# classes "_lognet" and "model_fit" (for class predictions):
	#
	# multi_predict()
	# multi_predict._lognet(penalty = NULL)
	# predict._lognet(multi = TRUE) <-- checks and sets penalty
	# predict.model_fit() <-- checks for extra vars in ...
	# predict_raw()
	# predict_raw._lognet()
	# predict_raw.model_fit(opts = list(s = penalty))
	# predict.lognet()

	# ------------------------------------------------------------------------------

Setting the penalty for logistic and multinomial regression with glmnet #863

Setting the penalty for logistic and multinomial regression with glmnet #863

Uh oh!

Conversation

hfrick commented Jan 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 21, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hfrick commented Jan 25, 2023 •

edited

Loading